{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "86920570-6be2-4aee-b24d-7d82cc923b5a",
   "metadata": {},
   "source": [
    "# Pearson Correlation Vs. Z-normalized Euclidean Distance\n",
    "\n",
    "It is [well understood](https://arxiv.org/pdf/1601.02213.pdf) that the z-normalized Euclidean distance, $ED_{z-norm}$, and the Pearson correlation, $PC$, between any two subsequences with length $m$ share the following relationship:\n",
    "\n",
    "$ED_{z-norm} = \\sqrt {2 * m * (1 - PC)}$\n",
    "\n",
    "Naturally, when the two subsequences are perfectly correlated (i.e., $PC = 1$), then we get:\n",
    "\n",
    "\\begin{align}\n",
    "    ED_{z-norm} ={}&\n",
    "        \\sqrt {2 * m * (1 - PC)}\n",
    "    \\\\\n",
    "    ={}&\n",
    "        \\sqrt {2 * m * (1 - 1)}\n",
    "    \\\\\n",
    "    ={}&\n",
    "        \\sqrt {2 * m * 0}\n",
    "    \\\\\n",
    "    ={}&\n",
    "        \\sqrt {0}\n",
    "    \\\\\n",
    "    ={}&\n",
    "        0\n",
    "    \\\\\n",
    "\\end{align}\n",
    "\n",
    "Similarly, when the two subsequences are completely uncorrelated (i.e., $PC = 0$), then we get:\n",
    "\n",
    "\\begin{align}\n",
    "    ED_{z-norm} ={}&\n",
    "        \\sqrt {2 * m * (1 - PC)}\n",
    "    \\\\\n",
    "    ={}&\n",
    "        \\sqrt {2 * m * (1 - 0)}\n",
    "    \\\\\n",
    "    ={}&\n",
    "        \\sqrt {2 * m * 1}\n",
    "    \\\\\n",
    "    ={}&\n",
    "        \\sqrt {2 * m}\n",
    "    \\\\\n",
    "\\end{align}\n",
    "\n",
    "In other words, the largest possible z-normalized distance between any pair of subsequences with length $m$ is $\\sqrt{2 * m}$. The maximum distance can never be bigger!\n",
    "\n",
    "Finally, when two subsequences are anti-correlated (i.e., $PC = -1$), then we get:\n",
    "\n",
    "\\begin{align}\n",
    "    ED_{z-norm} ={}&\n",
    "        \\sqrt {2 * m * (1 - PC)}\n",
    "    \\\\\n",
    "    ={}&\n",
    "        \\sqrt {2 * m * (1 - (-1))}\n",
    "    \\\\\n",
    "    ={}&\n",
    "        \\sqrt {2 * m * 2}\n",
    "    \\\\\n",
    "    ={}&\n",
    "        \\sqrt {4 * m}\n",
    "    \\\\\n",
    "    ={}&\n",
    "        2 * \\sqrt {m}\n",
    "    \\\\\n",
    "\\end{align}\n",
    "\n",
    "Note that while $2 * \\sqrt {m}$ (i.e., anti-correlated) is obviously larger than $\\sqrt {2m}$ (i.e., uncorrelated), it is basically impossible for a matrix profile distance to be \"worse\" than uncorrelated (i.e., larger than $\\sqrt {2m}$) due to the fact that it is defined as the distance to its one-nearest-neighbor. For example, given a subsequence `T[i : i + m]`, the matrix profile is supposed to return the z-norm distance to its one-nearest-neighbor. So, even if there existed another z-norm subsequence, `T[j : j + m]`, along the time series that was perfectly anti-correlated with `T[i : i + m]`, then any subsequence that is even slightly shifted away from location `j` must have a smaller distance than $2 * \\sqrt {m}$. Therefore, a perfectly anti-correlated subsequence would/could (almost?) never be a one-nearest-neighbor to `T[i : i + m]` especially if `T` is long.\n",
    "\n",
    "However, for normalizing pan matrix profiles,  $2 * \\sqrt {m}$ (anti-correlated) produces nearly identical visual results as $\\sqrt {2m}$ (uncorrelated). Thus, we chose to normalize with $2 * \\sqrt {m}$ (anti-correlated) as it is the most obvious and intuitive choice."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
